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ABSTRACT 


In the United States, approximately 37,000 individuals lose their lives in car accidents each year, representing a 
5.8% increase since 2014. The majority of these accidents, up to 90%, are caused by human errors. Autonomous 
vehicles have the potential to significantly reduce this high number of fatalities. One of the most important and 
widely adopted technologies in this field is line detection and lane keeping, which has been continuously improved 
since the 1980s. The desire to enhance road safety has led to the development of various systems that are integrated 


into vehicles. 


However, each new system adds complexity to the mathematical model and requires additional data representation 
to make accurate decisions. To address this, the goal is to create a system that mimics human driving based on 
predetermined rules, using artificial intelligence. As computer science progresses, one of its fundamental 
sub-branches, autonomous vehicles, becomes closely associated with computer vision. The primary objective of 
this technology is to design a system that can perform steering, braking, and acceleration independently. In this task, 
computer vision assists the system in detecting and identifying objects, while other algorithms handle the 


decision-making process. 
"2. Related Work 


There are various anticipated solutions for the problem at hand, including numerical methods, neural networks, 
reinforcement learning, convolutional neural networks (CNNs), and Q-learning. The two most significant 
approaches are numerical methods and CNNs, each with their own advantages and disadvantages. This paper 
introduces a simple, fast, and effective approach to address the problem. The numerical approach involves a 


meticulous analysis of the problem and can utilize either monocular or stereo vision. Many of these techniques 
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focus on identifying lane markers on the road, which are comparatively easier to detect. Typically, a reference point 


is necessary, such as a horizontal line in the image parallel to the x-axis. The lines should have sufficient thickness 
and form a shape resembling a parallelogram or approximation. Once the boundaries of the road are identified, the 
position of the vehicle can be determined using predefined camera calibration data. By accurately establishing the 
vehicle's position and employing advanced algorithms, the required steering angle can be computed. However, 
alternative methods similar in nature necessitate a level road, substantial distance, optical focus, pitch point, yaw 


angle, and height above the ground to perform intricate mathematical transformations. 


Another approach involves using CNNs, which require substantial data collection. This is done by recording 
commands given by human drivers and capturing images from onboard cameras. CNNs have revolutionized pattern 
recognition and can make accurate decisions based on past data. However, due to the high quality of CNNs, they 
require powerful hardware to run efficiently. Many solutions of this kind use multiple graphics processing units 
(GPUs) or dedicated hardware like the NVIDIA DRIVE platform, which significantly accelerates the learning rate 
and performance of a trained network. These solutions, however, often suffer from issues such as large size, 


complexity, high cost, and high-power consumption. 


One notable CNN solution is the end-to-end learning approach developed by nVidia, where the system learns to 
drive a car with minimal training data from a human driver, similar to the approach presented in this paper. 
However, a major drawback of this approach is the use of expensive hardware, such as the Nvidia Devbox and 


DRIVE platform, which can cost thousands of dollars. 


The objective of this paper's proposed solution is to create a more economical implementation of an autonomous 
driving AI system. This approach avoids the necessity for complex mathematical modeling and analysis of the 
problem. Additionally, it eliminates the requirement for power-intensive, high-end, and costly hardware for 


operation. The system can be trained with a limited amount of past data due to its straightforwardness. 


Figure 1. Overview of the robot 


“3, System Overview 


For this experiment, a specially constructed machine was utilized, incorporating Raspberry Pi and Arduino 


computers as the primary control units. The task of controlling the robot was performed by a separate computer, 
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with communication between the two systems established through a TCP/IP network. The robot was propelled by 


two wheels powered by steam and featured an additional independent wheel. It was equipped with five IR 
proximity sensors, with three facing forward and two positioned at the edges, along with a front-facing camera. The 
IR sensors were employed for detecting obstacles, while the camera was used to identify signs and traffic signals. 


The robot's appearance is depicted in Figure 1. 


Network 
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Figure 2. Connections of the electrical components of the system 


Effective communication between all modules is facilitated by clearly defined protocols, allowing for seamless 
updates and scalability when required. The system has been designed to handle a multitude of vehicles, diverse 
users, and numerous AI agents. This is achieved by implementing a concurrent and stateless configuration, enabling 
vehicles to be controlled by multiple instances of the AI agent concurrently. Furthermore, the utilization of a load 


balancer between these instances ensures efficient scalability of the system. 
= 4, Data Collection 


The training data was acquired through manual driving of the vehicle on a simulated road. Along with the usual 
road configuration featuring dark asphalt and white lines, the vehicle was also trained and evaluated on a darker 
road with white lines. The training data exclusively consisted of images captured from the front-facing camera 


while operating in manual mode. 


The collected data was stored as a set of attributes, with one parameter representing the command (direction of 
movement) sent to the vehicle, and the other parameter representing a video frame at the precise moment the 
command was given. 

= 5. Artificial Intelligence System 


The AI system functions as several state machines and has the task of issuing instructions to govern the vehicle. It 
employs a three-tier arrangement consisting of an input layer (camera image), a hidden layer, and an output layer 


(steering command). This configuration represents a basic fully connected neural network. The system's output 
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encompasses four potential options: three directions (forward, forward-left, forward-right), as well as a stop 


command. This design allows for seamless incorporation of forthcoming updates and alterations to particular 
algorithms. The implementation outlined in this paper operates in three distinct modes, as depicted in Figure 3. 


Initially, the vehicle operates in "Manual control" mode and can transition to other modes based on user input. 


User Action 


Figure 3. Autonomous vehicle operation modes 


In the "Semi-autonomous" mode, the state machine is straightforward, consisting of only two possible states. 
However, the more intriguing state machine is observed in the "Self-governing" mode, similar to autopilot, as 
depicted in Figure 4. It comprises four states, with "Free drive” serving as the initial and final state. In this state, the 
Artificial Neural Network (ANN) takes control of the vehicle. It continuously analyzes the input data and sensor 
readings for obstacle detection. Based on the identified objects or information from IR sensors, it determines 


whether to transition to the next state. 


If a directional signal is detected, the state machine switches to the "Course sign" state. In this state, the vehicle first 
checks for approaching traffic on the intersecting road. If the intersection is clear, the vehicle executes the 
appropriate command, such as making a turn in the desired direction. After completing this action, the state 


machine transitions back to the "Free drive" mode, returning control to the ANN. 
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Figure 4. Autonomous mode finite state machine 


6. Sign Detection 


The identification and recognition of traffic signs are crucial for self-driving vehicles. The OpenCV library offers 
the necessary tools for this task, including a trainer and a detector. In this case, classifiers were specifically 
implemented for stop signs, traffic lights, and directional signs (left, right, forward, and U-turn). Given the 
limitations of this approach, where each object requires its own classifier and scaling issues make it challenging to 


obtain sufficient training data, it was deemed suitable for proof of concept. 
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The behavior of the vehicle once a sign is detected is hard-coded. The distance to the object is measured for all types 


of signs, including stoplights. 
7. Conclusion 


One drawback of any AI system is its unpredictable behavior. However, this experiment demonstrated that the 
development of such systems can be relatively easy, especially for non-critical applications. Real-world 
implementation would require additional testing and safety protocols. Nevertheless, the demonstrated system was 
able to achieve all of this using minimal data and modest hardware. It successfully trained the system to drive in 
various conditions, including different times of day, urban and highway roads, and adapting to traffic regulations 


despite the presence of multiple vehicles. 
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